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1. INTRODUCTION 

The term of digital library (DL) can be defined as an organized collection of digital entities and 
objects, that can include the combination of audio, video, images, and text, embedded with the appropriate 
lifecycle for accessing, retrieving, selecting, creating, organization, maintenance and sharing [1], [2]. The use 
of DL is no longer considered as a lavish investment [3], [4]. It is a compulsory activity specially to support 
research and development, as well as postgraduate’s research activities [4]-[7]. A recent finding shows that 
there is an increase in the usage of DL in the aftermath of novel coronavirus 2019 (COVID-19) [8]. 

However, these recent surges of DL usages coincidently raise the issues of digital library 
engagement (DIL). Digital library engagement is operationalized as the extensive usage of the services, 
functions, and tools provided by the digital library. The digital library engagement extends the concept of 
information system usage. Previously, information system usage measure users based on a behavioral 
perspective, while digital library engagement measure usage from multiple perspectives: affective, cognitive, 
and behavioral perspectives [9]. Even though there are much research conducted on the DL, however, due to 
the extensive nature of the DL, more research is encouraged especially related to implementation, evaluation, 
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and engagement [4], [9]. Despite everything, the main concern and issues of the research surrounding the DL 
are mostly related to the traditional operational definition of information system usage, particularly to the 
behavioral perspective, indicating a scarce resource on the instrument measuring DL from the context of 
digital library engagement. 

Therefore, the aim of this study is to establish and confirm a research instrument to measure digital 
library engagement from the context of students enrolled for postgraduate studies at selected public 
universities in Malaysia. The next topics are structured as follows. The paper first explains the literature 
review underlying the topic of the study. Next, the method for developing and validating the instrument are 
presented. Subsequently, the findings of the study are discussed then concluded. 


2. LITERATURE REVIEW 
2.1. Digital library engagement 

User engagement is an important predictor of information system success [10]—[12]. The concept of 
user engagement goes beyond the traditional definition of information system usage by incorporating 
multiple indicators such as behavioral perspective, cognitive perspective, and affective perspective [4], [11], 
[13]. Previous studies show that engagement with information system influence: i) individual, 
ii) organizational, iii) quality of information system, and iv) end-user satisfaction [14]. 

DL engagement has been defined from several perspectives, such as e-commerce, information 
system, and library and information science. Baroudi et al. [10] views DL engagement as a combination of 
two factors; user participation and user involvement, in which user involvement is the closest concept to the 
concept of end-user engagement. From the perspective of e-commerce, engagement is mostly studied 
towards product retention and brand loyalty [15]-[17]. 

Exploring into the perspective of information systems (IS), researchers in IS area defined end-user 
engagement as the combined effect of user participation and user involvement [10], [14]. Hwang et al. [14] in 
their paper explain the concept of user engagement through the lens of user involvement and user 
participation. User involvement is the combination of the psychological or need-based state of a user on 
information system development and its process. On the other hand, user participation is the behavior of 
users that is observable during the development process. Among the two concepts, the concept of user 
involvement is considered as the strongest predictor of end-user engagement with the information system 
[10]-[12], [14]. 

Moving from the general perspective of engagement, the concept of engagement also applies to the 
field of library and information science (LIS). The dependency on digital information products leads to 
significant uptake of the digital library (DL) over the years [3]. Rahimi et al. [9] defined the digital library as 
the collection of library resources, objects, and services accessible through a digital platform. The resources 
include all databases subscribed by the library such as electronic journals, electronic books, video, and audio. 
In the context of Malaysian universities, the usage of the digital library is very common [4]-[7]. It is 
estimated that public universities academic libraries invested millions each year in order to ensure they have 
access to those digital library resources [4]-[7], [18]. Even though the cost of access to these resources are 
quite expensive, however, the digital library is considered as a necessary investment especially in the 
aftermath of the novel COVID-19 pandemic [3], [4] 

Even though many models and frameworks of DL usage have been developed over the years, yet the 
problem remains. Several researchers reported the underutilization of DL resources [7] [18]-[21]. For 
example, research shows that end-user such as students and academicians are more inclined towards free 
online resources compared to the digital library resources provided by their institution [6], [7], [18], [19], 
[22]. In another work, Tella et al. [19] highlights that search engine is the most used medium to access 
electronic information, indicating that DL resources have not been fully utilized by the users, indicating the 
underutilization of DL resources. 


2.2. Perceived benefits 

Perceived benefits are the potential impacts of digital library engagement. Perceived benefits can be 
operationalized as the degree that the engagement with the digital library contributed towards their emotional 
and functional benefits [9], [19]. Previous studies showed that deep engagement with an application system 
(such as the digital library) has been proven to contribute to the improvement and more benefits to the users 
[17], [23]-[25]. For this study, perceived benefits are evaluated from the context of functional and emotional 
benefits. 

Emotional benefits are operationalized as the degree of user satisfaction on the information system 
that helps them to fulfil their information needs [26]. An engaged individual tends to keep using the 
information system, and subsequently informing others about the efficiency and effectiveness of the 
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information system [4], [9], [22], [24], [26]. Several studies also found out that the concept of emotional 
benefits is contributed towards digital library engagement [3], [26]—[28]. 

On the other hand, functional benefits refer to the capability of DL to provide more tools and 
assistive capabilities for a postgraduate student to complete their task because of continuous engagement and 
interaction with the digital library. Continuous engagement with the digital library encourages and helps 
students to reduce the complexity of the information task through the usage of DL resources tools and 
services. Functional benefits help students to fulfil their potential and personal key performance indicators set 
up by their respective universities, such as publication and innovation exhibitions. 


2.3. Technological factors 

Technological factors are the extent of the technological characteristic of a digital library that 
influences digital library engagement. Technological factors are considered as prominent predictors of digital 
library engagement, especially in the domain of information systems, and library and information science 
[24], [29]. Technological factors influence extensive usage of digital library resources and features, as well as 
providing motivation roles towards continuing usage [30], [31]. 


2.4. Individual factors 

Individual differences are also known as individual factors. Individual differences are believed to be 
among the most important factor that should contribute towards digital library engagement; as well as 
indicators of human-computer interaction and the successful implementation of an information system [32]. 
Nelson [33] argue that the success of the information system (such as the digital library) depends on the 
interaction between the individual itself. Moreover, individual differences also contributed to subsequent 
usage and the success of the digital library system [34]—-[37]. 


2.5. Contextual factors 

Contextual factors or environmental factors can be defined as the uncontrollable external factors 
which have an influence on the achievement of user engagement and its subsequent impacts [9], [13]. Even 
though its importance is undeniable, there is a scarcity of research focusing on this element as the predictors 
of information system usage and digital library engagement [9]. Contextual factors refer to the stimuli from 
the outside of an individual, these stimuli may motivate or demotivate users from the object of interest. 


3. METHOD 

The study adopted and adapted the method to develop, and validated instruments based on the 
previous work of [35], [38]. The following subsection explained the selected process to develop and validate 
instruments measuring digital library engagement. Figure 1 shows the summary of the method. 


3.1. Identify predictors and impacts 

The first step is to identify the predictor variables and potential impacts of digital library 
engagement. A structured literature review (SLR) method was adopted, based on the previous similar work 
by [9], [39]. A literature search was conducted in three phases. First, the literature search starts with leading 
journals: Information Systems Research, International Journal of Information Management, Big Data and 
Society, European Journal of Information Systems, Library and Information Science Research, Journal of the 
Association for Information Science and Technology, and Journal of Enterprise Information Management. 
Second, a backward search for relevant literature was conducted. Third, a forward search was conducted to 
identify new research citing the previous works. As a result, a total of 14 variables were identified and 
categorized into 5 dimensions: technological (TEC), individual (IND), contextual (CON), digital library 


engagement (DIL), and perceived benefits (PEB). 
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Figure 1. Summary of research method 
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3.2. Instrument development 

Next, the items of the variables were developed by adapting and adopting from the previous 
relevant studies. The following Table 1 shows the summary of the instrument. The variables were grouped 
into 5 dimensions, namely technological factors, individual factors, contextual factors, digital library 
engagement, and perceived benefits. Technological factors comprise of service quality, information quality, 
and system quality. Individual factors consist of DL efficacy, domain knowledge, and attitude towards DL. 
On the other hand, contextual factors consist of task resources and task complexity. DL engagement is 
measured based on four constructs—focus attention, felt-involvement, novelty, and aesthetic. On the other 
hand, perceived benefits are measure based on emotional and functional benefits. 


Table 1. Sources of instrument 
Dimension Variable Item Sources 
Service quality (SEQ) [24], [27] 
Technological factors Information quality (INQ) s 
System quality (SYQ) 
DL efficacy (DLE) 
Individual factors Domain knowledge (DOK) 
Attitude towards DL (ADL) 
Task resources (TRE) 
Task complexity (TCO) 
Focus attention (FAT) 
Felt-involvement (FIV) 
Novelty (NOV) 
Aesthetic (AES) 
Emotional benefits (EMB) 
Functional benefits (FUB) 


[4042] 


Contextual factors 


Digital library engagement [4], [11], [31] 
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Perceived Benefits [26] 


3.3. Expert review process 

The instrument then was emailed to five (5) respective experts in the field of information 
management, information system management, e-commerce, and library and information science. The 
selection criteria of expert reviewers are: i) has a minimum of 10 years of academic experience, ii) holds a 
doctoral degree, and iii) has a related publication and seminar on the digital library, information system 
usage, and user engagement. Each expert was given 2 to 4 weeks to complete the assessment of the 
instrument. The instrument was evaluated using content clarity and content relevance. 


3.4. Face validity and reliability analysis 

Face validity was conducted by five (5) prospective respondents from one of the participating 
universities. These respondents will be excluded from the actual data collection process. Next, the instrument 
was validated by six (6) senior librarians specializing in the digital library. Following the face validity, a pilot 
test was conducted to determine the reliability of the instrument. 


3.5. Ethical approval 

Before actual data collection, an ethics application form was submitted to Universiti Teknologi 
MARA (UiTM) Research Ethics Committee for approval. The ethics approval was received on 29 October 
2019 and is valid until 1 December 2020. The approval number is REC/461/19. 


3.6. Measurement and structural model analysis 

Measurement and structural model analysis were conducted upon completion of reliability analysis. 
Data was collected from 492 respondents, selected through convenience sampling, and analyze based on 
structural equation modelling-partial least square (SEM-PLS). The use of PLS-SEM enables higher-order 
constructs to be analyzed more accurately compared to the use of regression analysis. Convenience sampling 
was used due to difficulties accessing respondents’ responses due to the confidentiality of students’ records. 
In relation to sampling size, this study utilizes the rules-of-ten to determine the minimum and maximum 
sample size for the study [43]. 


4. FINDINGS 

The following subsections will explain the findings and output of the study. The findings are 
grouped into assessment of expert review process, content clarity and content relevance, face validity, 
reliability analysis, and measurement and structural model analysis. The iterative process was developed to 
confirm the validity and reliability of the instrument developed for the purpose of the study. 
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4.1. Expert review process 

A total of five (5) experts were selected for the study. All five (5) experts hold a doctoral degree and 
have more than 10 years of academic experience. Furthermore, all experts were designated as senior 
lecturers, indicating their level of experience with the research process in the academic setting. The 
instrument was sent, and all experts were given three weeks to return the instrument to the researcher. The 
instrument is evaluated using the content validity index (CV-I) of [44]. 


4.2. Content clarity and content relevance (CV-1) 

The following Table 2 show the summary of content clarity and content relevance, as rated by the 
selected experts. The content clarity and content relevance (CV-I) is the degree of the instrument having the 
appropriate number of items and is considered as an important procedure in instrument development [44]. 
The result shows that both content clarity and content relevance produce a strong acceptance among the 
experts. However, some minor modifications were made to the final instrument in response to the comment 
and suggestion from the expert review process. 


Table 2. Content clarity and content relevance 
Expert 1 Expert2_ Expert3 Expert4 Expert 5 
Content Clarity Proportion Relevant 0.84 0.97 1 0.97 0.98 
Content Relevance 0.9 0.98 1 0.96 1 


4.3. Face validity 

Face validity was conducted with five (5) prospective respondents from the participating 
universities. These respondents are excluded from the total size population. Respondents were asked to 
answer and evaluate their understanding of the instrument. All respondents agreed that the instrument is 
sufficient and understandable, therefore no changes are made to the instrument. Further up face validity was 
also conducted with 6 senior librarians specializing in the digital library. Vice versa, all respondents agreed 
that the instrument is sufficient, clear, and relevant to the topic of interest. 


4.4. Reliability analysis 

Following the completion of the face validity process, a pilot instrument was developed to test the 
reliability of the instrument. An online instrument using Google Form was developed and emailed to 
100 students at one of the participating universities. A final total of 85 valid responses were received. The 
number of responses indicating a response rate of 85%. Next, the response was analyzed using Cronbach’s 
alpha coefficient in order to determine whether the instrument is reliable. Table 3 shows the final analysis 
result of the Cronbach’s alpha coefficient. The result shows that the values of Cronbach’s Alpha are between 
0.763 to 0.940, surpassing the minimum value of 0.7 as suggested by [45], [46]. 


Table 3. Reliability analysis 


Dimension Construct Cronbach's Alpha 
Service Quality 0.851 
Technological Information Quality 0.906 
System Quality 0.832 
Digital Library Efficacy 0.763 
Individual Domain Knowledge 0.845 
Attitude towards Digital Library 0.935 
Cöntežtüal Task Resources 0.774 
Task Complexity 0.810 
Focus Attention 0.926 
Sa . Felt Involvement 0.898 
Digital Library Engagement Aesthetic 0.940 
Novelty 0.879 
: ; Emotional Benefits 0.940 
Perceived Benetit Functional Benefits 0.876 


4.5. Measurement and structural model analysis 
To confirm the measurement and model analysis, quantitative research was conducted. With the 
help of the assistant registrar of each university, a survey email invitation was sent to all Malaysian research 
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universities postgraduate students at the selected universities. These methods were proven effective because 
the assistant registrar have the access to all students’ data and was willing to help in distributing the 
questionnaire through the email invitation. 

As a result, a valid 492 responses were received, indicating a valid sample size based on the 
ten-times rule [43], [47]. Data were coded, perform data cleaning before being analyzed using SmartPLS 
version 3.2.8. During measurement model analysis, a total of two runs was conducted. One variable, DLES 
was removed due to low factor loading (0.295), indicating a valid removal of the construct (1.6%). Next, the 
model was assessed for convergence and discriminant validity. Results show that all variables factor loading 
ranging from 0.412 to 0.869 while the average variance extracted (AVE) value ranging from 0.505 to 0.855, 
therefore sufficient to confirm convergent validity. 

Next, heterotrait-monotrait ratio (HTMT) test was performed on the assessment model. The findings 
shows that there are no values above 0.9, therefore, indicating discriminant validity has been ascertained. 
Thus, it is confirmed that measurement model analysis has been achieved and the instrument is deemed 
sufficient to proceed to the structural model analysis. 

The structural model analysis was conducted following the completion of the measurement model 
analysis. The following Table 4 shows that all hypotheses of the study are accepted, and the model is 
empirically validated. As a result, an instrument consisting of 61 items underneath 5 main dimensions and 
14 constructs. The subsequent Figure 2 shows the final model of the study. 


Table 4. Hypothesis testing results 
The Relationship Original Sample (O) Sample Mean (M) Standard Deviation (STDEV) __t-value p-values 


CON > DIL 0.194 0.197 0.045 4.284 0.000 
DIL > PEB 0.763 0.764 0.022 34.852 0.000 
IND > DIL 0.349 0.344 0.066 5.265 0.000 
TEC > DIL 0.296 0.3 0.067 4.438 0.000 


Note: *t > 1.645, p-value > 0.05, ** t > 2.58, p-value < 0.01 


TECHNOLOGICAL 
Service Quality 
Information Quality 
System Quality 
DIGITAL LIBRARY 


INDIVIDUAL ENGAGEMENT. PERCEIVED BENEFITS 
Digital Library Efficacy Focus Attention Emotional 


Domain Knowledge Felt Involvement Functional 
Attitude Towards DL Aesthetic 
Novelty 


CONTEXTUAL 
Task Resources 
Task Complexity 


Figure 2. Final model 


5. DISCUSSION 

Studies on DL engagement have become one of the most important agendas for the nation. With the 
vision of moving towards industrial revolution 4.0, the National Library of Malaysia (PNM) has introduced a 
lot of initiatives to foster digital information technology in the management of information resources. 
However, previous research shows that the use of DL resources was underutilized, indicating that there is 
significant ignorance on the use of subscribed resources. Moreover, previous studies also show that the 
contextual factors have received little attention over the years. Findings within other domains show that 
contextual factors do have a significant relationship with DL engagement. However, the empirical evidence 
of this relationship is scarce and almost non-existent in the field of library and information science (LIS). 
Other than that, there is an insufficiency of research on the concept of DL engagement in the perspective of 
its determinants and impacts. Therefore, this study provides empirical evidence of the relationship of DL 
engagement in relation to its determinants and impacts. 

Moreover, one of the practical contributions of this study is the development of a comprehensive 
instrument to measure DL engagement in relation to its determinants and impact. In the context of this study, 
the instrument that was developed was a survey consisting of 61-items. The instrument was developed by 
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adapting and adopting previous research instruments with the combination of new items especially in the 
context of contextual factors. The conceptual framework was used as guidance for the development of the 
questionnaire. Practitioners, researchers, and academicians specializing in the field of information system 
management, user engagement, and information system usage may utilize this instrument to further enhance 
the knowledge on the digital library engagement. 

Besides, the result of this study also confirmed the previous similar studies in user engagement, such 
as the work of [4], [5], [8]-[11], [13]-{15], [19], [25]-[27]. On top of it, this study extends the previous 
studies by confirming and validating the concept of digital library engagement in relation to its determinants 
and impacts, as well as providing valid items in the context of postgraduate students. The findings of this 
study can be generalized to other countries that have a similar setting with Malaysia, such as Thailand, 
Myanmar, Philippines, and Singapore. Moreover, the instrument can be adapted to other fields of studies 
such as information systems (IS), e-Commerce, and Psychological. Besides, the move from traditional into 
online learning cause the necessary dependency on prompt engagement between educators and learners due 
to pandemic of novel coronavirus 2019 [48]. 

In relation to method, the use of the content clarity and content relevance (CV-I) index of [44] 
shows that the measurement of the instrument is more reliable and accurate. Instead of having an open-ended 
measurement, the use of CV-I enables an expert to evaluate the items on the individual level and from a 
predefined perspective. It also helps the researcher to determine the selection and removal criteria of items as 
a result of the CV-I measurement. The digital library engagement underlies researchers’ reputation and 
affinity for scientific work, which are, respectively, a key motivational driver and a potential feature of 
researchers' motivation for high-quality publications [49], [50]. In practice, the implementation of the 
proposed tool for measuring digital library engagement will contribute to higher scientific results of 
researchers/universities and, as a result, higher university ratings and accreditation scores. 


6. CONCLUSION 

The conclusion of the quantitative study signal that the study has achieved its purpose to develop 
and validate the instrument measuring digital library engagement. Therefore, it can be concluded that the 
instrument consisting of 61-items are sufficient to be used in a research project related to digital library 
engagement. The instrument will be of interest to practitioners, academicians, government policymakers, and 
software vendors. 

This study is not without limitations. First, we only conducted a quantitative study from a single 
context of the library and information science. Future studies may test the instrument from various fields of 
study such as e-commerce, information system, management, and psychological. Second, we only used a 
minimal number of second-order variables. Future studies may enhance the model by incorporating more 
second-order variables such as socioeconomic factors and organizational factors. 
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